The ability to recognize patterns is a major step towards the development of artificial systems and mobile entities, such as robots, that are capable of performing perceptual tasks that currently only biological systems can perform. Speech and visual pattern recognition are two areas in which conventional computers are seriously deficient. In an effort to develop artificial systems that can perform these and other tasks, new methods based on neural models of the brain are being developed to perform perceptual tasks. These systems are known variously as neural networks, neuromorphic systems, learning machines, parallel distributed processors, self-organizing systems, or adaptive logic systems. Whatever the name, these models utilize numerous nonlinear computational elements operating in parallel and arranged in patterns reminiscent of biological neural networks. Each computational element or "neuron" is connected via weights or "synapses" that typically are adapted during training to improve performance. Thus, these systems exhibit self-learning by changing their synaptic weights until the correct output is achieved in response to a particular input. Once trained, neural nets are capable of recognizing a target and producing a desired output even where the input is incomplete or hidden in background noise. Also, neural nets exhibit greater robustness, or fault tolerance, than conventional sequential computers because there are many more processing nodes, each with primarily local connections. Damage to a few nodes or links need not impair overall performance significantly.
There is a wide variety of neural net models utilizing various topologies, neuron characteristics, and training or learning rules. Learning rules specify an internal set of weights and indicate how weights should be adapted during use, or training, to improve performance. By way of illustration, some of these neural net models include the Perceptron, described in U.S. Pat. No. 3,287,649 issued to F. Rosenblatt; the Hopfield Net, described in U.S. Pat. Nos. 4,660,166 and 4,719,591 issued to J. Hopfield; the Hamming Net and Kohohonen self-organizing maps, described in R. Lippman, "An Introduction to Computing with Neural Nets", IEEE ASSP Magazine, April 1987, pages 4-22; and the Generalized Delta Rule for Multilayered Perceptrons, described in Rumelhart, Hinton, and Williams, "Learning Internal Representations by Error Propagation", in D. E. Rumelhart and J. L. McClelland (Eds.), Parallel Distributed Processing; Explorations in the Microstructure of Cognition. Vol. 1: Foundations. MIT Press (1986).
Neural networks are discussed in each of the following U.S. Patents, the contents of each of which are hereby incorporated by reference in order to more fully describe the state of the art to which the subject invention pertains:
U.S. Pat. No. 4,807,168 issued Feb. 21, 1989 (Moopenn et al.).
U.S. Pat. No. 4,884,216 issued Nov. 28, 1989 (Kuperstein).
U.S. Pat. No. 4,903,226 issued Feb. 20, 1990 (Tsividis).
U.S. Pat. No. 4,918,617 issued Apr. 17, 1990 (Hammerstrom et al.).
U.S. Pat. No. 4,950,917 issued Aug. 21, 1990 (Holler et al.).
U.S. Pat. No. 4,951,239 issued Aug. 21, 1990 (Andes et al.).
U.S. Pat. No. 4,962,342 issued Oct. 9, 1990 (Mead et al.).
U.S. Pat. No. 4,972,187 issued Nov. 20, 1990 (Wecker).
U.S. Pat. No. 4,994,982 issued Feb. 19, 1991 (Duranton et al.).
U.S. Pat. No. 4,996,648 issued Feb. 26, 1991 (Jourjine).
U.S. Pat. No. 5,003,490 issued Mar. 26, 1991 (Castelaz et al.).